Quiz on MapReduce
Test your understanding of concepts related to the design of the MapReduce system via a quiz.
Question 7
To tackle stragglers, we introduced the reassignment of tasks to other idle workers. What if the reassigned tasks also become stragglers? How do you think our system will handle it?
Our system handles stragglers independently of whether they are first-layer stragglers or not. If our system marks a worker as a straggler, it will reassign its tasks to other idle workers, regardless of the straggler running a reassigned task itself. The system will keep reassigning the task unless it marks it as done.
If most jobs finish very slowly, a manager might not run a lot of re-executions. The manager detects a straggler by comparing the timespan of a job with the average finish time of already-completed jobs. If the average finish time is very high, the manager might not be able to detect them until their running time goes above the average.
Nevertheless, having many stragglers might indicate a more severe issue, such as a failing network switch, and might need human intervention to recover. If there are a lot of stragglers on a specific server, the manager can stop assigning that specific server any future tasks.
7 of 7
Concluding MapReduce
Introduction to Spark
